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Abstract 

Ensembles of elimate models are eommonly used to improve elimate predietions and assess the uneer- 
tainties assoeiated with them. Weighting the models aeeording to their performanees holds the promise of 
further improving their predietions. Here, we use an ensemble of deeadal elimate predietions to demonstrate 
the ability of sequential learning algorithms (SLAs) to reduee the foreeast errors and reduee the uneertain- 
ties. Three different SLAs are eonsidered, and their performanees are eompared with those of an equally 
weighted ensemble, a linear regression and the elimatology. Predietions of four different variables-the sur- 
faee temperature, the zonal and meridional wind, and pressure-are eonsidered. The spatial distributions of 
the performanees are presented, and the statistieal signifieanee of the improvements aehieved by the SLAs 
is tested. Based on the performanees of the SLAs, we propose one to be highly suitable for the improvement 
of deeadal elimate predietions. 
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I. INTRODUCTION 


Global circulation models are the main tools used to simulate future elimate eonditions. There 
are two main praetiees by whieh to initialize these models that represent predietions for two dif¬ 
ferent time seales. The first praetiee eorresponds to long-term elimate projeetions. In this type of 
simulation, the elimate models are initialized in the pre-industrial era (aka uninitialized runs) and 
integrated forward in time (usually until 2100). In these simulations, the atmospherie eomposition 
in the past is set aeeording to observations, while for the future, several representative eoneentra- 
tion pathways [|T]|, eorresponding to different seenarios of atmospherie eomposition ehanges, are 
used. These elimate simulations are expeeted to provide information about the response of the 
elimate system to different emission seenarios by predieting the ehanges in the long-term averages 
(10 years and more) and the statisties of elimate variables, under different atmospherie eomposi¬ 
tion seenarios m. 

The seeond praetiee, whieh is eonsidered in this work, is near-term (deeadal) elimate predie¬ 
tions intended to provide information on the dynamies of the elimate system in time seales shorter 
than those of signifieant ehanges in the atmospherie eoneentration and the response time of the 
elimate system to sueh ehanges. In this praetiee, the elimate models are initialized with observed 
eonditions elose to the predietion period. The expeeted information from these simulations is the 
dynamies of the monthly to deeadal averages of elimate variables [|3]-0, whieh is of great impor- 
tanee for elimate serviees 0. Reeent studies have demonstrated a potential deeadal predietion 
skill in different regions and for different physieal proeesses lffl[5l[8l-fT0l. 

Despite their relatively short term, deeadal elimate predietions are still aeeompanied by large 
uneertainties, and new methods to improve the predietions and reduee the assoeiated uneertainties 
are of great interest. One of the main approaehes to improving elimate predietions is to eom- 
bine the output from an ensemble of elimate models. This approaeh has two known advantages 
eompared with single model predietions. First, it was shown that the ensemble average generates 
improved predietions [ITTI - fT^ ; seeond, the distribution of the ensemble member predietions ean 
provide an estimate of the uneertainties. However, the simple average of elimate simulations does 
not aeeount for the quality differenees between the ensemble members; therefore, it is expeeted 
that weighting the ensemble members based on their past performanees will inerease the foreeast 
skill. 

Uneertainties in elimate predietions ean be attributed to three main sourees. The first is internal 
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variability, that is, uncertainties due to different initial eonditions (either different initialization 
times or different initialization methods) that were used to run a speeifie model. The seeond souree 
is model uncertainties due to different predietions of different models. The third souree is forcing 
scenario uncertainties due to different scenarios assumed for the future atmospheric composition 
m. The eontribution of these sourees to the total uneertainty of the elimate system varies with 
the prediction lead time and is also spatially, seasonally and averaging-period dependent [fTSl . 
It was shown that for global and regional deeadal elimate predietions, seenario uneertainties are 
negligible compared to the first two sources iTTTlfTgl . 

There are two eontributions to the internal variability-variability due to different starting eondi¬ 
tions and variability due to different initialization methods. Uneertainties due to different starting 
conditions stem from the chaotic nature of the simulated climate dynamics and cannot be redueed 
using the ensemble approaeh. However, uneertainties due to different initialization methods and 
the model variability can be reduced by weighting the members of the ensemble. The total reduc¬ 
tion of the uneertainty depends on the relative eontribution of these sourees to the total uneertainty. 

Bayesian inferenee is one of the methods that have been used in the past to weight an ensemble 
of climate models. The main part of this method is the calculation of the posterior density, whieh 
is proportional to the produet of the prior and the likelihood. The Bayesian method optimizes the 
probability density function (PDF) of the climate variable to the PDF of the data during a learning 
period and uses it for future predietions. It does not assign weights to the elimate models; instead, it 
gives an estimation for the PDF of the predieted climate variable. Bayesian inferenee has been used 
extensively for projeetions of future elimate [120] - |2^ and also for near-term elimate predietions 
ll29l [^ . The use of Bayesian inferenee has redueed the uneertainties of the elimate projeetions 
and improved their near-term predietions. However, this method relies on many assumptions 
regarding the distribution of the elimate variables that are not always valid, making the Bayesian 
inferenee subjective and variable-dependent. 

A seeond, and more eommon, method that has been used to improve elimate predietions is 
linear regression [|3TI - I42l . The linear regression method does not assign weights to the ensem¬ 
ble members but rather attempts to find a set of eoeffieients such that the scalar product of the 
veetor of eoeffieients and the veetor of the model predietions yields the minimal sum of squared 
errors relative to past observations. The same set of eoeffieients is then used to produce future 
predietions. As a eonsequenee, the regression ean be used only for deterministie predietions, that 
is, the linear eombination of the models is calculated to produce better predietions, but there is no 
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straightforward method to estimate the assoeiated uneertainties. Similarly to the Bayesian method, 
the regression method also relies on a few inherent assumptions, sueh as the normal distribution 
of the predietion errors (therefore, defining the optimal eoeffieients as those minimizing the sum 
of squared errors) and the independenee of the ensemble member predictions. 

Sequential learning algorithms (SLAs, also known as online learning) [|4^ weight ensemble 
members based on their past performances. These algorithms were shown to improve long-term 
climate predictions [l44l|45l and seasonal to annual ozone concentration forecasts EH El- More 
recently, it was shown that decadal climate predictions of the 2m-temperature can be improved 
using SLAs and can even become skillful when the climatology is added as a member of the 
ensemble [|48l . The SLAs have several advantages over the other ensemble methods described 
above. First, they do not rely on any assumption regarding the models and the distribution of 
the climate variables. In addition, the weights assigned to the models can be used for model 
evaluation and the comparison of different parameterization schemes or initialization methods. 
Third, the weighted ensemble provides not only predictions but also the associated uncertainties. 
All these characteristics suggest that the SLAs are suitable for the improvement of various climate 
variable predictions. 

Here, we test the performances of SLAs in predicting the, previously investigated, 2m- 
temperature and three additional climate variables-namely, the zonal and meridional components 
of the surface wind and the surface pressure. The results of the CMIP5 ll^ decadal experiments 
constitute the ensemble, and the NCEP reanalysis data [[50l are considered as the observations. 
The performances of the SLAs are compared with those of the regression method. The compar¬ 
ison with the Bayesian method is not straightforward and is not included here. We also study 
the effects of different learning periods and different bias correction methods on the SLA perfor¬ 
mances. This paper is organized as follows. In Section |I^ we present the data that we used in 
this study, including the models and the reanalysis data. In addition, we discuss the different bias 


correction methods that we used. In Section IIL we describe the SLAs and the regression fore¬ 
casting methods as we implemented them. We also provide the details of the climatology that we 
derived from the reanalysis data. In Section|V[ we present the predictions of the different forecast¬ 
ing methods. We also evaluate their global and regional performances based on their root mean 
square errors (RMSEs). The global and regional uncertainties of the predictions of the different 
forecasting methods are presented in Section |V^ The weights assigned by the SLAs to the differ¬ 
ent models and to the climatology (all the members of the ensemble) are presented in Section [VTH 
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The results are diseussed and summarized in Section IVTTTi 


II. MODELS AND DATA 

The decadal experiments were introduced to the Coupled Model Intercomparison Projects 
(CMIP) multi-model ensemble in its fifth phase (CMIP5). The objective of these experiments 
is to investigate the ability of climate models to produce skillful future climate predictions for a 
decadal time scale. The climate models in these experiments were initialized with interpolated 
observation data of the ocean, sea ice and atmospheric conditions, together with the atmospheric 
composition [|^ . The ability of these simulations to produce skillful predictions was not investi¬ 
gated widely, but it was shown that they can generate skillful predictions in specific regions around 
the world 0 S HOl [Ml ISMS . 

The CMIP5 decadal experiments were initialized every five years between 1961 and 2011 for 
10-year simulations, with three exceptional experiments that were extended to 30-year simulations. 
One of these 30-year experiments was initialized in 1981 and simulated the climate dynamics till 
2011. The output of four variables from this experiment is tested here-surface temperature, zonal 
and meridional surface wind components, and surface pressure. In what follows, we analyze the 
monthly means of these variables. 

Table |I] shows the eight climate models included in our ensemble. The decadal experiments 
of the CMIP5 project include a set of runs for each of the models, differing by the starting date 
and the initialization scheme used. We chose, arbitrarily, the first run of each model. As long 
as the model variability is the main source of uncertainty, the choice of the realization should 
not be significant for our analysis. Indeed, it was found that, in the CMIP5 decadal experiments, 
the model variability is the main source of uncertainty, independent of the prediction lead time, as 
long as the predictions are not bias corrected. Bias correction reduces mainly the model variability; 
however, the contribution of the model variability remains important lITSl . 

The NCEP/NCAR reanalysis data [l50l were used as the observation data for the learning and 
for the evaluation of the forecasting methods performances. We are aware of other reanalysis 
projects [|56l[57]l : however, we selected the NCEP based on its wide use (note that the assessment 
of the quality of the different reanalysis projects is subjective and is beyond the scope of this 
paper). The effects of using different reanalysis data are left for future research. 
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TABLE I: Model Availability 


Institute ID 

Model Name 

Modeling Center (or Group) 

Grid (lat X Ion) 

BCC 

BCC-CSMl.l 

Beijing Climate Center, China Meteo¬ 
rological Administration 

64 X 128 

CCCma 

CanCM4 

Canadian Centre for Climate Mod¬ 
elling and Analysis 

64 X 128 

CNRM-CERFACS 

CNRM-CM5 

Centre National de Recherches Me- 

teorologiques / Centre Europeen de 

Recherche et Eormation Avancees en 

Calcul Scientifique 

128 X 256 

EASG-IAP 

EGOAES-S2 

EASG, Institute of Atmospheric 

Physics, Chinese Academy of Sciences 

108 X 128 

IPSE* 

IPSE-CM5A-ER 

Institute Pierre-Simon Eaplace 

96X96 

MIROC 

MIROC5 

MIROC4h 

Atmosphere and Ocean Research In¬ 
stitute (The University of Tokyo), Na¬ 
tional Institute for Environmental Stud¬ 
ies, and Japan Agency for Marine- 

Earth Science and Technology 

128 X 256 

320 X 640 

MRI 

MRI-CGCM3 

Meteorological Research Institute 

160X320 


* not available for U and V components of wind 


A. Bias correction 

The predictions made by the climate models often suffer from inherent systemic errors [[58l . 
and it is common to apply bias correction methods to the model outputs before analyzing them. 
For long-term climate projections, this procedure is more straightforward because of the available 
reference period. Bias correction in decadal climate predictions is not trivial not only because 
there is no clear reference period but also because some of these experiments are known to have a 
drift from the initial condition to the model’s climatology during the first years of the simulation 

m. 
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Here, two bias eorreetion methods and the original data were eonsidered. The original data 
without any bias eorreetion is noted as no correction. The first bias eorreetion method eorresponds 
to subtraeting from eaeh model results their average during the learning period and adding the 
elimatologieal average (the average of the NCEP/NCAR reanalysis data for the same period). 
This method is noted as average correction. The seeond bias eorreetion method eorresponds to 
subtraeting from eaeh model and for eaeh ealendar month the eorresponding average during the 
learning period and adding the NCEP/NCAR reanalysis average for that ealendar month during 
the same learning period. This method is noted as climatology correction. The two bias eorreetion 
methods deseribed above do not aeeount for the explieit time dependenee of the bias. However, it 
is reasonable to assume that for deeadal elimate predietions, the bias does not ehange eonsiderably 
with time. 

III. FORECASTING METHODS 

In this work, we eonsider three sequential learning algorithms (SLAs), introdueed below. More 
thorough deseriptions of the SLAs ean be found in Ref. [|4^ and in Ref. [|59l . We also eonsider 
the linear regression (REG) OTll method in order to eompare the performanees of the SLAs to the 
well-known regression method. The elimatology (CLM) is eonsidered here as the threshold for 
skillful predietions. Eor elarity, the equations that deseribe the foreeasting methods omit the spatial 
indiees. However, the foreeasting sehemes were applied to eaeh of the grid eells independently, 
thereby allowing the spatial distribution of the weights (or the eoeffieients in the ease of the REG) 
and the referenee elimatology. 


A. The EWA and the EGA 

The SLAs use an ensemble of experts (elimate models), eaeh of whieh provides a predietion 
for a future value of a elimate variable, to provide a foreeast of the elimate variable in terms of the 
weighted average of the ensemble. The proeess is sequentially repeated with the weights of the 
models being updated, after eaeh measurement, aeeording to their predietion skill. We divide the 
period of the model simulations into two parts. The first part is the learning (or training) period 
whose data is used to update the model weights in the manner deseribed above, and the seeond 
part is used for validating and evaluating the forecaster performanee. At the end of the learning 
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period, the learning ends and the weights generated by the SLA in the last learning step are used 
to weight the predietions of the elimate models during the validation period. 

The deviation of the predietion of model E, fE,u from the observed value, yt, determines the 
loss function, l{fE,t,yt), at time t. Similarly, the loss funetion of the forecaster (the SLA) is 
determined by the deviation of its prediction, pt, from the observed value at time t. The loss 
function is the metric used to evaluate the models performances. In our study, we define the 
loss function as the square of the deviation, namely, l{fE,t,yt) = ifE,t — UtY for model E and 
KPt,yt) = {Pt -ytY for forecaster. 

The output of the Exponentiated Weighted Average (EWA), the first SEA described here, at 
time t is the set of the weights of the models in the ensemble: 


„,,EWA ^ 

^E,t — 6 


( 1 ) 


where 77 is a positive number representing the learning rate of the forecaster and Zt is a normal¬ 
ization factor. The EWA prediction at time t is defined below: 


N, 


Pt 


E ,,,EWA r 

^E,t-1 ■ JE,t, 


E=1 


( 2 ) 


where Ng is the number of models in the ensemble. 

The second SEA considered here is the Exponentiated Gradient Average (EGA). The EGA 
assigns the weights according to the following rules: 


EGA _ 
^E.t 


EGA 


WeT' = — -WEjZi-e 


( 3 ) 


where If t is the gradient of the forecaster loss function with respect to the weight of model E at 
time t — 1. The mathematical definition of If t is provided below: 


.EGA _ aiiPt .»)_ 2. (pf“ 




(4) 


where the prediction of the EGA, pf^^, is defined similarly to the prediction of the EWA: 


Pf^^ - 


Ng 

L 

E=1 


W 


EGA 


UA r 
,t-l ■ JE,t- 


( 5 ) 


An important difference between the EWA and the EGA is the fact that under ideal conditions 
and stationary time series, the EWA converges to the best model in the ensemble, while the EGA 
converges to the observations [|48l. 
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Note that for the first learning step, one has to assign initial weights to the models. Without any 
a priori knowledge of the models performanees, the natural ehoiee is to assign equal weights to all 
the models. If the hierarehy of the models is known, it is possible to assign their initial weights 
aeeordingly. 

The learning rate, r], was optimized by seanning a wide range of values and using the value that 
resulted in the minimal RMSE during the learning period. However, we added a restrietion that the 
maximal ehange in the weight of eaeh of the models, between two learning steps, will be smaller 
than the weight of eaeh model in an equally weighted ensemble-namely, 1/Nf,. This restrietion 
was added to ensure the stability of the weights. The metrie that we used for this optimization is 
defined below: 



( 6 ) 


where 0 represents the Heaviside theta funetion, and RMSE is the root mean squared error of 
\he forecaster during the n time steps of the learning period. The RMSE for a grid eell (i, j) is 
eonventionally defined. 


n 



The value of r] that minimizes M was found using a reeursive seareh within a very wide range 
of values restrieted only by the maehine preeision. The optimization was done for eaeh grid eell 
separately. 

B. The Learn -a algorithm 

The basie form of the EWA was modified to explieitly allow switehing between experts. This 
switehing improves the performanee of the SLA when dealing with nonstationary time series. The 
fixed-shared algorithm introdueed in Ref. llbOll is defined by the following rules: 



( 8 ) 


where 


K(E, £•; a) = (1 - a) ■ S{E, E‘) + ■ (1 - S{E, E')). 


( 9 ) 


Here, a E [0,1] is the switehing rate parameter, and 5(-, ■) is the Kroneeker delta. 
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The fixed-share algorithm was extended in Ref. [[59l by also learning the optimal switching 
rate parameter, a. This modified SLA is known as the Leam-a algorithm (LAA). In the LAA, the 
algorithm scans a range of switching rates, j G 1 ,A'q,, and assigns weights to each value of 
Oj based on a loss per alpha function. It (aj) = — log (^e=i («j) j. The weights are 

updated sequentially for both the switching rate and the experts. The updating rule for the weight 
of a specific value, aj, is provided below: 

( 10 ) 


The updating rule for the weight of expert E, given aj, is provided below: 


iVe 


z, (aj) je. 

The prediction at time t, is the weighted average of the experts and the different values of a. 

Afe Nc 

W^i-1 («j) ■ Wej-i (tti) ■ fE,t- (12) 

E=1j=l 

Here, we adopted a discretization of a to optimize the LAA performance 


C. Regression 

The linear regression algorithm considered here is described in Ref. llTTl . In this algorithm, the 
forecast is a linear combination of the climate model predictions as described below: 

N 

= y + '^aE{fE,t-JE)- (13) 

E=1 

Here, y = (1/n) XltLi Vt is the temporal mean of the observed values during the learning period 
(similarly, f ^ is the temporal mean value of the predicted values by expert E during the learning 
period), and are the regression coefficients minimizing the sum of squared errors during the 
learning period, G, which is defined below: 

n 

G = ^{Pt-ytf, (14) 

t=i 

where n is the number of time steps in the learning period. The algorithm that we used to minimize 
G involved the elimination of models that were linearly dependent on the other models in the 
ensemble. 
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D, Climatology 


The climatology is defined here as the monthly averages of the observed conditions during the 
learning period. Namely, 

ni 

Cm ^ ^ yt,m ( 15 ) 

t=l 

where yt,m is the observed value in month m G [1,12] of year t (t is measured in years from the 
beginning of the simulations), and rii is the duration of the learning period in years (for simplicity, 
we assume here that both the learning and the validation periods span an integer number of years). 
The twelve months of the climatology were replicated to match the duration of the validation 
period; that is, 

CLMt^m = Cm-, (16) 

for t E [ni + 1, ni + n 2 ] is the duration of the validation period in years). The climatology 
is often considered as the threshold for a skillful prediction, i.e., di forecaster that outperforms the 
climatology is considered skillful. 


IV. EVALUATION METRICS 

Two main evaluation metrics are used here: the average error, quantified by the RMSE of each 
of the forecasters, and the variability of the ensemble predictions, characterized by their standard 
deviation, the STD. The global averages of the RMSE and the STD are calculated by weighting 
each grid cell by the fraction of the earth’s surface it spans. The precise details are provided here 
for clarity. During the validation period, the RMSE of each forecaster was calculated for each 
grid cell (because all the climate variables studied here are two-dimensional, each grid cell has 
two indices, (f, j)) from the time series of the forecast and the observations. Then, the global 
area-weighted average of the RMSE {RMSE gaw) was calculated as detailed below: 

RMSEgAW = if/S^Earth) ^i,j RMSE (j, j), (17) 

where AEarth is the total earth’s surface area, and A{i, j) is the area spanned by the {i,j) grid cell. 
In what follows, we will present both the spatial distribution of the RMSE and its global average. 

Similarly to the RMSE, the variance of the ensemble predictions was calculated for each of the 
grid cells at each time point and then averaged over time during the validation period. The square 
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root of this temporally averaged varianee is what we define here as the STD of eaeh grid eell. The 
mathematieal definition of the STD is provided below: 


STD(i,]) = 


\ 


n N 




(18) 


t=l E=1 

The global area-weighted average was then ealeulated: 


STDgAW = 


(19) 




The skill of the forecasters was measured by eomparing their RMSE and STD to those of some 
other reference forecaster. For eonvenienee, we define below the RMSE skill seore, Rrefjct- 


RMSEref - RMSE fa 
^ 1 {RMSEref + RMSE fa)' 


( 20 ) 


The indiees ref and fct are used to identify the forecasters whose skills are eompared. Similarly, 
we define below the STD skill seore, Srefjct- 

STD ref - STD fa 


S, 


ref, fct — I 


(STDref + STDfa)' 


( 21 ) 


Unless otherwise speeified, we used the elimatology as the reference forecaster for Rref.fct and the 
equally weighted ensemble as the referenee/orecarter for Sref,fct- Note that the skill seores are 
defined sueh that a forecaster with a smaller RMSE than the referenee forecaster has a positive 
Rref,fct seore, and similarly, n forecaster with a smaller STD (i.e., smaller uneertainty) than the 
referenee/ore colter has a positive Sref,fct seore. 


V. PREDICTIONS 

A. Global 

The simplest measure of the performanee of the forecasters is the global average of the root 
mean squared error, RMSEgaw- Figure shows the RMSEgaw of tho validation period for the 
five different/orerouter5, EWA, EGA, REG, EAA and GEM, and the different learning periods. 
The rows (from top to bottom) eorrespond to the surfaee temperature, zonal wind, meridional 
wind, and pressure, respeetively. The eolumns (from left to right) eorrespond to no bias eorreetion, 
average bias eorreetion, and elimatology bias eorreetion, respeetively. The data is provided in 
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No correction 


Average correction 


Climatology correction 


S3 3.5 
B 



Surface temperature 


Learning Period [Years] 
Zonal wind 


Learning Period [Years] 

Meridional wind 


Learning Period [Years] 
Surface pressure 


It 




5 10 15 20 25 

Learning Period [Years] 


M 1.5 
Q 

M 1.4 
to 

1 1.3 
1.2 





jIi 

J lll i 




Learning Period [Years] 

Zonal wind 


Learning Period [Years] 

Meridional wind 


Learning Period [Years] 
Surface pressure 






Learning Period [Years] 

Zonal wind 



Learning Period [Years] 
Meridional wind 



Learning Period [Years] 
Surface pressure 



Learning Period [Years] 


Learning Period [Years] 


FIG. 1: Globally averaged RMSE. RMSEgaw for the five forecasting methods (EWA, EGA, REG, 
EAA and GEM), learning periods of 5, 10, 15, 20 and 25 years and the four climate variables (surface 
temperature, two wind components and pressure). The ensemble used by the, forecasters does not include 
the climatology. The left panels correspond to no bias correction, the middle panels correspond to average 


bias correction, and the right panels correspond to climatology bias correction (see Section IIA for the 
details of the different bias correction methods). 


Tables 1-4 of the Supplementary Information. The deeadal elimate simulations eonsidered here 
span a 30-year period that is split sueh that the first part is used for learning and the second part is 
used for the evaluation of the performances; that is, for the five-year learning period, the validation 
period is the next 25 years, and for the 10-year learning period, the validation period is the next 
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20 years, etc. The RMSEqaws of the individual models are not presented because they are much 
higher than those of the. forecasters. The RMSEqaw of the equally weighted ensemble is much 
lower than those of the models, but it is also too high to be included within the scale shown in 
Fig. The bias correction that resulted in the smallest RMSEqaw^ is the climatology correction, 
which is described in Section iHAl 

Figure shows that the climatology outperforms all the other forecasters, for all the learning 
periods and bias correction methods studied here. Therefore, we added the climatology as an 
expert to the ensemble. Unless otherwise specified, the following results were derived from an 
ensemble including the climatology as an additional expert [|48ll . 

Figure shows the same results as Figure for an ensemble that includes the climatology. In 
addition, the initial weight assigned to the climatology was 0.5, whereas the initial weight of all 
the other models was 0.5/(A^e — 1) (^e — 1 is the number of the models excluding the climatology). 
This higher initial weight of the climatology was motivated by its superior performance (as shown 
in Fig. [^and [|48l|). The data that was used to generate Fig. [^is provided in Tables 5-8 of the 
Supplementary Information. 

The results of Fig. [^show that the best predictions are obtained using 20 years of learning and 
different bias correction methods for different variables and different forecasters. The fact that 
the RMSEgaw is minimized after 20 years of learning can be related to two factors: i) for short 
learning periods, there is a longer prediction period and, therefore, a larger RMSEgaw'^ ii) for the 
25-year learning period, the time lead from the initialization to the prediction period is long, and 
in addition, the short five-year prediction period does not represent the climate variability over a 
time scale of 25 years (the duration of the learning period). The 20 years of learning also ensures 
that the learning period extends well beyond the drift of the models. In Table we detail the 
bias correction that resulted in the smallest RMSEgaw for esich forecaster and for each climate 
variable. In what follows, we will present only the results of these bias corrections and 20 years of 
learning. We find that all the SLAs have a lower or equal RMSEgaw than the climatology for the 
surface temperature and wind components. For the surface pressure, only the LAA outperforms 
the climatology. We also see that, for most climate variables, the RMSEgaw^ of the EWA and 
the climatology are almost equal. This is not a coincidence; it reflects the fact that the EWA 
tracks the best model, which in most grid cells, is the climatology. The two other SEAs reduce 
the RMSEgaw below that of the climatology by extracting information from the other models 
in the ensemble. The EAA outperforms the EGA for short learning periods (< 15 years) and 
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FIG. 2: Globally averaged RMSE with climatology. RMSEgaw for the five forecasting methods (EWA, 
EGA, REG, EAA and GEM), learning periods of 5, 10, 15, 20 and 25 years and the four climate variables 
(surface temperature, two wind components and pressure). The ensemble used by the forecasters includes 
the climatology. The left panels correspond to no bias correction, the middle panels correspond to average 
bias correction, and the right panels correspond to climatology bias correction. 


for all learning periods in the predietions of the surface pressure. This better performance can be 
attributed to the design of the LAA for the learning of nonstationary data. The poorer performance, 
relative to the climatology, of most of the forecasters (except for the LAA) in the prediction of 
the surface pressure is not fully understood. However, we found that for the surface pressure, 
the variability between the models is often larger than its seasonal variability, while all the other 
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climate variables eonsidered here show seasonal variabilities that are larger than the variabilities 
between the models. It is also possible that the model predietions of the monthly mean surfaee 
pressure are worse than the predietions of the other elimate variables. 

TABLE II: The optimal bias correction for each forecaster and each climate variable. T, U, V, and P 
denote the surface temperature, zonal wind, meridional wind and pressure, respectively, nbias, bias and 
mbias correspond to the no correction, average correction and climatology correction, respectively. 


Forecaster 

Climate variable 


T 

U 

V 

P 

EGA 

nbias 

nbias 

nbias 

bias 

EWA 

mbias 

nbias 

nbias 

nbias 

EAA 

bias 

bias 

nbias 

bias 

REG 

nbias 

nbias 

nbias 

nbias 

AVG 

mbias 

mbias 

mbias 

mbias 


B. Regional 

The RMSEgaw is eonvenient beeause it aims to quantify the performanees of the forecasters 
using only one number. However, often the more seientifieally and praetieally relevant information 
are the spatial distributions of the RMSE. In this subseetion, the spatial distribution of the fore¬ 
caster performanees will be investigated using the Rrefjct metrie defined above. This metrie will 
allow us to eompare the performanees of the different forecasters and, in partieular, to eompare 
their performanees to that of the trivial/oreca^ter-the elimatology. The statistieal signifieanee of 
the improvement aehieved by the forecasters was tested by introdueing the null hypothesis that 
the temporal distribution of Rrefjct is symmetrie around 0. Grid eells in whieh the hypothesis 
was rejeeted with a 90% eonfidenee level in favor of a better/oreca^ter performanee are marked 
with white dots. Similarly, grid eells in whieh the hypothesis was rejeeted in favor of a poorer 
forecaster performanee are marked with blaek dots. Grid eells in whieh the data does not provide 
enough evidenee to rejeet the null hypothesis are not marked. 

Figure depiets the spatial distributions of Rclm,ega (upper left panel), Rclm,ewa (upper 
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FIG. 3: Surface temperature RMSE skill score. Upper left panel: EGA, upper right panel: EWA, lower 
left panel: EAA and lower right panel: REG. Positive values correspond to a smaller RMSE than the 
climatology and vice versa. White circles represent significant improvement and black circles represent a 
significantly poorer performance. 

right panel), Rclm,laa (lower left panel) and Rclm,reg (lower right panel) for the surfaee tem¬ 
perature. This figure better elarifies the origin of the EGAs superior performanee over the other 
forecasters (as seen from the surfaee temperature panels, the 20-year learning period bins of Fig. 
[^. The largest variability is observed for Rclm,ega and the smallest variability for Rglm,laa- 
While the LAA shows a positive skill seore over large regions, the seore is relatively low, refleeting 
a small improvement in the predietion eompared with the elimatology. For the EGA, on the other 
hand, we see that over regions in the North Atlantic, South America, central Africa, and Oceania, 
there is a large improvement relative to the climatology, while in regions in the East China Sea, the 
South Atlantic Ocean and the Eastern Central Pacific Ocean, there is a much poorer performance 
compared with the climatology. The regression forecaster shows a poorer performance compared 
with the climatology (negative skill score) over most of the globe. All the forecasters show a pos¬ 
itive skill over regions in North Africa, Asia and North America, suggesting that the models are 
capable of capturing deviations from the climatology in these regions. 
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FIG. 4: Surface zonal wind RMSE skill score. Upper left panel: EGA, upper right panel: EWA, lower 
left panel: EAA and lower right panel: REG. Positive values correspond to a smaller RMSE than the 
climatology and vice versa. White circles represent significant improvement and black circles represent a 
significantly poorer performance. 

The spatial distribution of the RMSE skill seore for the zonal and meridional wind eomponents 
are shown in Figures and respeetively. Both wind components have similar characteristics. 
The EGA shows a similar distribution of the skill for the wind components to that found for the 
surface temperature. The EWA and the EAA show almost zero skill over most of the globe due 
to the fact that they both assign a very high weight to the climatology and a very small weight to 
the other models. Although the improvement relative to the climatology is small, it was found to 
be statistically significant in many regions. The REG shows a poorer performance compared with 
the climatology over most of the globe. 

Eigure shows the spatial distribution of the surface pressure Rclm,ega (upper left panel), 
Rclm,ewa (upper right panel), Rclm,laa (lower left panel) and Rclm,reg (lower right panel). 
The EGAs performance for the surface pressure is poor compared with its performance for the 
other variables. Earge regions in the Pacific and Indian Oceans show a larger RMSE of the EGA 
than the climatology, while in some regions in the Atlantic Ocean, North Euro-Asia, Greenland 
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FIG. 5: Surface meridional wind RMSE skill score. Upper left panel: EGA, upper right panel: EWA, 
lower left panel: EAA and lower right panel: REG. Positive values correspond to a smaller RMSE than the 
climatology and vice versa. White circles represent significant improvement and black circles represent a 
significantly poorer performance. 

and the South Pacific the EGA shows a better performance than the climatology. The EWA and 
EAA assign a very high weight to the climatology and, therefore, show an RMSE skill score close 
to zero. However, the small improvement achieved by the EAA is statistically significant over 
most of the globe. The REG shows a poorer performance than the climatology over most regions, 
with some exceptions in the central Atlantic Ocean and the Arabian Peninsula. 

The EGA shows the highest RMSE skill score over most of the globe for the surface tempera¬ 
ture and wind components, while the EAA shows the highest score for the surface pressure. There 
are several regions (such as the North Atlantic, North Indian Ocean and North Euro-Asia) where 
the SEAs seem to provide a smaller RMSE than the climatology. This suggests that at least some 
of the models capture processes that result in a deviation from the climatology and that the SEAs 
are capable of tracking these models. 


19 




















































































FIG. 6: Surface pressure RMSE skill score. Upper left panel: EGA, upper right panel: EWA, lower 
left panel: EAA and lower right panel: REG. Positive values correspond to a smaller RMSE than the 
climatology and vice versa. White circles represent significant improvement and black circles represent a 
significantly poorer performance. 

VI. UNCERTAINTIES 

The RMSE is an important measure of the quality of the predietions; however, the uneer- 
tainties assoeiated with the predietions of the forecasters are erueial for a meaningful assessment 
of the predietions quality. The uneertainties are quantified here using the standard deviation of 
the ensemble. A natural reference for comparing the variance of the ensemble weighted by the 
forecasters is the variance of the equally weighted ensemble that represents no learning. It was 
mentioned earlier that the linear regression does not assign weights to the models in the ensemble 
but rather attempts to find the linear combination of their predictions that minimizes the sum of 
squared errors. Therefore, in this section, we will compare the uncertainties of the three SLAs 
and the equally weighted ensemble, denoted here as AVR. Our analysis proceeds similarly to the 
analysis of the RMSE-, first we present the globally averaged standard deviation, STD caw ^ and 
then we present the spatial distribution of the STD skill score. 
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FIG. 7: Globally averaged STD. STDcaw for the three SLAs (EWA, EGA, and EAA) and for the 
equally weighted ensemble, AVR, for learning periods of 5, 10, 15, 20 and 25 years and the four climate 
variables (surface temperature, two wind components and pressure. The ensemble used by the forecasters 
(and AVR) does not include the climatology. The left panels correspond to no bias correction, the middle 
panels correspond to average bias correction, and the right panels correspond to climatology bias correction 
(see Section [ITAI for the details of the different bias correction methods). 

A. Global 

Figure|^shows STDcaw of the EGA, EWA, LAA and AVR for different learning periods and 
for the four elimate variables eonsidered in this study. The results of Fig. were derived from 
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an ensemble that does not inelude the elimatology. The four left panels eorrespond to no bias 
eorreetion, the four middle panels eorrespond to average bias eorreetion and the four right panels 
eorrespond to elimatology bias eorreetion. The data is provided in Tables 9-12 of the Supplemen¬ 
tary Information. As expeeted, the more detailed the bias eorreetion, the smaller the uneertainty 
beeause it is assoeiated with the anomaly rather than with the aetual predietion. We also notiee that 
without bias eorreetion, the EWA has the smallest STDgaw, while with bias eorreetion, the LAA 
shows the smallest STDgaw- Both the EWA and the EAA are expeeted to have lower STDs be¬ 
eause they traek the best models. With no bias eorreetion, all the SEAs show smaller uneertainties 
than the equally weighted ensemble, while for the elimatology bias eorreetion, the EGA shows a 
higher STDgaw than the AVR. This suggests that in large regions, the EGA assigns high weights 
to models spanning a broad range of predieted values. In addition, we notiee that the STDgaw is 
smaller for longer learning periods, or more preeisely, for shorter predietion periods, as expeeted. 
The reduetion of STDgaw is more signifieant for the EAA beeause the longer learning allows it 
to better traek the elimatology despite the built-in switehing rate. 

Eigurej^is similar to Eig. |7]but for an ensemble that ineludes the elimatology. The data used 
to generate Eig. [^is provided in Tables 13-16 of the Supplementary Information. It is apparent 
that in this ease, the STDgaw of all the SEAs is smaller than that of the AVR, and for the longer 
learning periods, it is mueh smaller. The large reduetion in the STDgaw of the EWA and EGA 
is elearly assoeiated with the faet that they traek the elimatology in most regions (beeause it is 
the best expert in these regions). The STDgaw of the EGA is also redueed beeause it assigns a 
high weight to the elimatology in many regions, but it still assigns signifieant weights to the other 
models; therefore, it has a larger STDgaw than the other SEAs. The STDgaw of the equally 
weighted ensemble does not ehange mueh beeause the elimatology is only assigned a weight of 
1 /Ae, and in most regions, the elimatology is spanned by the other models. 

B. Regional 

The uneertainty has a large spatial variability. We foeus on the 20-year learning period and the 
ensemble that ineludes the elimatology. The STD skill seore shows the average temporal vari¬ 
ability of the ensemble weighted by the forecasters eompared with that of the equally weighted 
ensemble during the validation period. Eigurej^ shows the spatial distribution of the surfaee tem¬ 
perature STD skill seore for the three SEAs. The EGA has a positive STD skill seore (smaller 
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FIG. 8: Globally averaged STD with climatology. STDgaw for tho three SLAs (EWA, EGA, and EAA) 
and for the equally weighted ensemble, AVR, for learning periods of 5, 10, 15, 20 and 25 years and the 
four climate variables (surface temperature, two wind components and pressure). The ensemble used by 
th& forecasters (and AVR) includes the climatology. The left panels correspond to no bias correction, the 
middle panels correspond to average bias correction, and the right panels correspond to climatology bias 
correction (see Section [nA| for the details of the different bias correction methods). 

STD than the equally weighted ensemble) in most of the globe, but there are many regions in 
whieh its STD is signifieantly larger than that of the AVR. The EWA reduees the STD over most 
of the globe exeept for the tropics. This reduction of the STD stems from the high weight as¬ 
signed to the climatology. In many regions, the Savr,ewa is around 2, which reflects an almost 
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FIG. 9: Spatial distribution of the surface temperature STD skill score. Upper panel: EGA, middle panel: 
EWA, and lower panel: EAA. Positive values correspond to a smaller STD than the equally weighted 
ensemble and vice versa. White circles represent a statistically significant reduction of the STD and black 
circles represent a statistically significant increase of the STD relative to the STD of the equally weighted 
ensemble. 


vanishing STD of the EWA. The LAA also shows a smaller STD than the AVR over most of the 
globe except for small regions in the tropics. Similarly to the EWA, the reduction of the uncer¬ 
tainties achieved by the EAA stems from the high weight assigned to the climatology. However, 
one can see that the Savr,laa is smaller than the Savr,ewa, which reflects a lower weight of the 
climatology and higher weights of the other models due to the built-in switching rate in the EAA. 

Eigures [T0| and 11 show the STD skill score of the EGA, EWA and EAA for the zonal and 
meridional wind components. Eor the wind components, all the SEAs show significant reductions 
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FIG. 10: Spatial distribution of the surface zonal wind STD skill score. Upper panel: EGA, middle panel: 
EWA, and lower panel: EAA. Positive values correspond to a smaller STD than the equally weighted 
ensemble and vice versa. White circles represent a statistically significant reduction of the STD and black 
circles represent a statistically significant increase of the STD relative to the STD of the equally weighted 
ensemble. 


of the STD over most of the globe. The EGA and LAA show larger STDs in some small regions 
in the tropics. The results suggest that all the SLAs assign a high weight to the climatology, with 
the EWA almost fully converging to it, while the EGA and EAA extract information from the 
models as well. 


Eigure 12 shows the surface pressure STD skill score for the three SEAs. The EWA and 
EAA show positive skill scores over the entire globe, and the EGA only shows negative skill in a 
very small region in Oceania. The EWA fully converges to the climatology and has a vanishing 
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FIG. 11: Spatial distribution of the surface meridional wind STD skill score. Upper panel: EGA, middle 
panel: EWA, and lower panel: EAA. Positive values correspond to a smaller STD than the equally weighted 
ensemble and vice versa. White circles represent a statistically significant reduction of the STD and black 
circles represent a statistically significant increase of the STD relative to the STD of the equally weighted 
ensemble. 

STD (resulting in an Savr,ewa around 2 over the entire globe). The LAA also converges to the 
climatology, but due to the built-in switching probability, the weight assigned to the climatology 
is slightly smaller than 1 , and accordingly, the Savr,laa is slightly smaller than 2 . 
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FIG. 12: Spatial distribution of the surface pressure STD skill score. Upper panel: EGA, middle panel: 
EWA, and lower panel: EAA. Positive values correspond to a smaller STD than the equally weighted 
ensemble and vice versa. White circles represent a statistically significant reduction of the STD and black 
circles represent a statistically significant increase of the STD relative to the STD of the equally weighted 
ensemble. 

VII. EGA WEIGHTS 

Some of the results above regarding the skill of the forecasters were explained by the weights 
assigned to the elimatology. Due to its superior performance, compared with each of the models in 
the ensemble, it is expected that the SLAs would assign it a high weight. However, assigning too 
high a weight to the climatology implies that the forecaster is not capable of capturing deviations 
from the climatology due to the physical processes captured in the models. IdedlXy, forecasters 
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FIG. 13: Spatial distribution of the weight assigned to the climatology by the EGA forecaster for (a) 
surface temperature, (b) surface zonal wind, (c) surface meridional wind and (d) surface pressure. 


should balance between the smaller RMSE of the elimatology and the additional information 
available from the other models. 

Figures and 15 show the spatial distribution of the weight assigned to the elimatology, 

for eaeh of the four elimate variables, by the EGA, EWA and EAA, respeetively. The weights in 
these figures correspond to the weights assigned at the end of the 20-year learning period (i.e., the 
weights used for the predictions). The eolorbar was set to emphasize the differenees. The EWA 
assigns the climatology weights close to 1 over the entire globe for the surface wind eomponents 
and pressure. Eor the surface temperature, there are large regions in the tropics, close to the 
North Pole and along the eoast of Antarctica where the weight of the elimatology is not the only 
dominant expert. Similar patterns are observed for the EAA; however, the weight assigned to the 
climatology here is never 1 beeause this SEA is based on the fixed-share SEA that is designed to 
have a finite switehing probability. Both the weights assigned by the EWA and those assigned by 
the EAA stem from the faet that these SEAs are designed to track the best expert, whieh in our 
ensemble turns out to be the elimatology over most of the globe. 

The EGA assigns a lower weight than the EWA and EAA to the elimatology over most of the 
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FIG. 14: Spatial distribution of the weight assigned to the climatology by the K forecaster for (a) 
surface temperature, (b) surface zonal wind, (c) surface meridional wind and (d) surface pressure. 

globe for all the climate variables considered here. For the surface temperature, only in some 
regions (mostly in the eastern Pacific Ocean) are the predictions of the EGA dominated by the 
climatology. For the surface wind components, the regions dominated by the climatology are 
somewhat larger. The weight assigned to the climatology by the EGA for the surface pressure 
shows a much larger variability (note the nontrivial color map) than the weight assigned for the 
other variables. This variability also resulted in a somewhat poorer performance by the EGA in 
the predictions of this variable. This different performance for the surface pressure may be related 
to the lower quality of the data for this variable. Unlike the EWA and the EAA, the EGA is not 
designed to track the best expert but rather to track the measurements. Therefore, the lower weight 
assigned to the climatology suggests that useful information can be extracted from the models, 
and their ability to capture some of the processes affecting the climate dynamics in decadal time 
scales can be quantified by the weight assigned to them by the EGA. 
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FIG. 15: Spatial distribution of the weight assigned to the climatology by the l^AA forecaster for (a) 
surface temperature, (b) surface zonal wind, (c) surface meridional wind and (d) surface pressure. 

VIII. SUMMARY AND DISCUSSION 

An ensemble of climate models is known to improve climate predictions and to help better 
assess the uncertainties associated with them. In this paper, we tested five different methods to 
combine the results of the decadal predictions of different models-EWA, EGA, EAA, REG and the 
equally weighted ensemble. The first ihxQQ forecasters represent learning algorithms that weight 
the ensemble models according to their performances during a learning period. The REG attempts 
to find the linear combination of the model predictions that minimizes the sum of squared errors 
during the learning period, and the equally weighted ensemble represents no learning. We tried 
different learning periods and found the 20-year learning experiment to be the most promising. 
This learning period ensures that the learning exceeds well beyond the drift of the models. The 
RMSE and STD are smaller than those of shorter learning periods, and the results suggest that 
the lead time (the time from the initialization of the models) has a small effect. The predictions of 
the surface temperature, wind and pressure were studied, and their qualities were assessed. 

The simple average was shown to have larger errors and larger uncertainties than the. forecasters 
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that used a learning period to weight/combine the model predictions. The linear regression showed 
smaller errors than the equally weighted average. When no bias correction was applied to the data 
and the ensemble did not include the climatology, the errors of the regression were even smaller 
than those of the learning algorithms. However, in the more relevant ensemble that includes the 
climatology, the errors of the linear regression were higher than those of the learning algorithms. 
This poorer performance is associated with the basic assumptions of the linear regression and its 
oversimplified method to linearly combine the model predictions. The SLAs do not rely on these 
assumptions and use more advanced methods to weight the models, resulting in smaller errors. 
The REG method does not weight the models but rather finds an optimal linear combination of 
them; therefore, there is no straightforward method to estimate the uncertainties associated with 
the linear regression predictions. The EWA and the EAA were found to be more appropriate in 
cases in which tracking of the best model is of interest. The climatology outperformed all the other 
models; therefore, the EWA and the EAA converged to it over most of the globe and for all the 
four climate variables. Tracking the best model (by the EWA and EAA) was shown to result in too 
small uncertainties and thus in overconfident predictions. Eor the purpose of improving decadal 
climate predictions, we found the EGA to be more appropriate because it showed both the ability 
to reduce the errors and to provide more meaningful estimates of the uncertainties. 

Although the globally averaged RMSE of the EGA is only a few percentage points smaller 
than that of the climatology, it was shown to be statistically significant. In addition, we found 
that in many regions, the improvement is larger. The spatial distribution of the EGA performance 
showed that it is skillful over large continuous regions. This finding suggests that the models were 
able to capture some physical processes that resulted in deviations from the climatology and that 
the EGA enabled the extraction of this additional information. Similarly, the large regions over 
which the climatology outperforms the forecasters may suggest that physical processes, associated 
with the climate dynamics affecting these regions, are not well captured by the models. The 
EGA performance was much poorer for the surface pressure than for the other variables. This 
poorer performance might be related to the quality of the models output or to the large fluctuations 
of this variable. The better predictions of the EWA and EAA for the surface pressure cannot 
be considered significant because their performance is similar to that of the climatology. The 
reduction of the uncertainties is much more substantial than the reduction of the errors and can 
reach to about 60 — 70%, globally. The uncertainties considered here are only those associated 
with the model variability within the ensemble. The internal uncertainties, scenario uncertainties 
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and other sources of uncertainty were not studied here. 

The results presented here are in agreement with previous results (see Ref. [jll and references 
therein). However, in this work, monthly means were considered, whereas in previous works, the 
averages of longer periods, which have smaller fluctuations, were considered. A predictive skill 
of the EGA can be observed in the North Atlantic, in the North Indian Ocean and in some regions 
in the Pacific Ocean. In addition, the EGA showed predictive skill over many land areas, such 
as North Euro-Asia, Greenland, and, to some extent, also the Americas. The results suggest that 
learning algorithms can be used to improve climate predictions and to reduce the uncertainties 
associated with them. 
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Appendix A: Supplementary Information 

This Supplementary Information provides the globally averaged root mean square errors for the 
(MiQXQni forecasters and different learning periods. The results provided here were used to select 
the optimal bias correction method for each forecaster and each climate variable. In addition, the 
globally averaged standard deviations of the ensemble weighted by the diifcxcnt forecasters are 
provided. 
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TABLE III: The surface temperature RMSEgaw for tho different/oreca^ter^ and bias correction methods. 


Forecaster 

Bias correction 

Eearning period 



5 

10 

15 

20 

25 

EGA 

No correction 

1.4432 

1.4267 

1.4177 

1.3716 

1.3876 

Avg. correction 

1.3624 

1.3394 

1.346 

1.2972 

1.3143 

Clm. correction 

1.3257 

1.2584 

1.2614 

1.2123 

1.2312 

EWA 

No correction 

1.4812 

1.4687 

1.4755 

1.4287 

1.4552 

Avg. correction 

1.3811 

1.356 

1.3573 

1.3158 

1.3349 

Clm. correction 

1.3397 

1.2632 

1.2556 

1.2086 

1.2317 

EAA 

No correction 

1.4764 

1.4898 

1.5233 

1.5062 

1.5472 

Avg. correction 

1.3595 

1.368 

1.3948 

1.3717 

1.4013 

Clm. correction 

1.3193 

1.2825 

1.2906 

1.2593 

1.288 

REG 

No correction 

1.4895 

1.3632 

1.3376 

1.2922 

1.301 

Avg. correction 

1.4789 

1.3607 

1.3381 

1.2902 

1.2989 

Clm. correction 

1.5446 

1.3241 

1.2868 

1.2304 

1.2415 

AVG 

No correction 

1.7776 

1.7918 

1.8111 

1.8046 

1.8422 

Avg. correction 

1.4152 

1.4118 

1.4184 

1.3904 

1.4194 

Clm. correction 

1.2898 

1.2475 

1.2472 

1.2038 

1.2338 

GEM 


1.2393 

1.1994 

1.2213 

1.1881 

1.1956 
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TABLE IV: The zonal surface wind RMSEgaw for tho differentand bias correction methods. 


Forecaster 

Bias correction 

Eearning period 



5 

10 

15 

20 

25 

EGA 

No correction 

1.7814 

1.7694 

1.7765 

1.7618 

1.8004 

Avg. correction 

1.6905 

1.6806 

1.6829 

1.6698 

1.7076 

Clm. correction 

1.7468 

1.672 

1.6589 

1.628 

1.667 

EWA 

No correction 

1.8047 

1.7979 

1.8027 

1.7946 

1.8312 

Avg. correction 

1.7021 

1.6934 

1.695 

1.6806 

1.7221 

Clm. correction 

1.7589 

1.6802 

1.6634 

1.6297 

1.6738 

EAA 

No correction 

1.8114 

1.8259 

1.8557 

1.8678 

1.9199 

Avg. correction 

1.7105 

1.7234 

1.7499 

1.75 

1.8074 

Clm. correction 

1.758 

1.7014 

1.7077 

1.6873 

1.7431 

REG 

No correction 

1.7633 

1.7197 

1.7134 

1.699 

1.7366 

Avg. correction 

1.7296 

1.6865 

1.6801 

1.6637 

1.7034 

Clm. correction 

1.8124 

1.6892 

1.6645 

1.6277 

1.6662 

AVG 

No correction 

1.8947 

1.8982 

1.9094 

1.9126 

1.9665 

Avg. correction 

1.7182 

1.7112 

1.7188 

1.7072 

1.7519 

Clm. correction 

1.7429 

1.6697 

1.6629 

1.6312 

1.677 

GEM 


1.6285 

1.5719 

1.569 

1.5323 

1.5692 
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TABLE V: The meridional surfaee wind RMSEqaw for tho difier&nt forecasters and bias eorreetion 


methods. 


Foreeaster 

Bias eorreetion 

Eearning period 



5 

10 

15 

20 

25 

EGA 

No eorreetion 

1.4532 

1.4502 

1.4575 

1.4488 

1.4701 

Avg. eorreetion 

1.3923 

1.3839 

1.3897 

1.3808 

1.3987 

Clm. eorreetion 

1.4251 

1.3607 

1.3553 

1.3336 

1.3474 

EWA 

No eorreetion 

1.4722 

1.4703 

1.4765 

1.4698 

1.4937 

Avg. eorreetion 

1.4033 

1.3941 

1.398 

1.3892 

1.4102 

Clm. eorreetion 

1.4355 

1.3647 

1.3577 

1.3355 

1.3528 

EAA 

No eorreetion 

1.4756 

1.493 

1.5169 

1.5275 

1.5599 

Avg. eorreetion 

1.4065 

1.4162 

1.4371 

1.4412 

1.47 

Clm. eorreetion 

1.4336 

1.3829 

1.3896 

1.3792 

1.4019 

REG 

No eorreetion 

1.4412 

1.4078 

1.4081 

1.3956 

1.4149 

Avg. eorreetion 

1.4171 

1.3839 

1.3831 

1.3692 

1.3867 

Clm. eorreetion 

1.4673 

1.3651 

1.3526 

1.3279 

1.3401 

AVG 

No eorreetion 

1.5319 

1.5349 

1.5424 

1.544 

1.5708 

Avg. eorreetion 

1.4145 

1.4109 

1.4174 

1.4114 

1.4323 

Clm. eorreetion 

1.4185 

1.36 

1.3563 

1.3355 

1.352 

GEM 


1.3352 

1.2853 

1.2876 

1.2617 

1.2731 
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TABLE VI: The surface pressure RMSEgaw for the different/oreca^ter^ and bias correction methods. 


Forecaster 

Bias correction 

Eearning period 



5 

10 

15 

20 

25 

EGA 

No correction 

3.7837 

3.735 

3.7369 

3.7327 

3.8133 

Avg. correction 

2.6627 

2.6388 

2.6592 

2.6314 

2.7475 

Clm. correction 

2.7774 

2.628 

2.6278 

2.5765 

2.682 

EWA 

No correction 

3.9589 

3.9483 

3.9562 

3.9528 

4.043 

Avg. correction 

2.689 

2.6561 

2.6687 

2.6458 

2.7556 

Clm. correction 

2.8111 

2.6444 

2.6335 

2.5824 

2.6838 

EAA 

No correction 

4.1878 

4.1653 

4.2056 

4.2313 

4.3497 

Avg. correction 

2.7081 

2.7503 

2.8186 

2.8327 

2.9609 

Clm. correction 

2.8254 

2.7417 

2.7802 

2.7597 

2.8885 

REG 

No correction 

2.8237 

2.6976 

2.6875 

2.6583 

2.7572 

Avg. correction 

2.824 

2.6978 

2.6877 

2.6584 

2.7574 

Clm. correction 

3.035 

2.7184 

2.6809 

2.6158 

2.7049 

AVG 

No correction 

6.4757 

6.4749 

6.4933 

6.4998 

6.5855 

Avg. correction 

2.6786 

2.659 

2.6839 

2.6639 

2.7813 

Clm. correction 

2.7506 

2.6155 

2.6208 

2.5701 

2.6785 

GEM 


2.5746 

2.4531 

2.4606 

2.3935 

2.4933 


42 




TABLE VII: The surface temperature RMSEgaw for the diffemntforecasters and bias correction methods. 
The climatology is included in the ensemble. 


Forecaster 

Bias correction 

Eearning period 



5 

10 

15 

20 

25 

EGA 

No correction 

1.229 

1.1919 

1.2146 

1.1562 

1.1516 

Avg. correction 

1.2204 

1.1909 

1.2088 

1.1681 

1.17 

Clm. correction 

1.2616 

1.2095 

1.2202 

1.177 

1.183 

EWA 

No correction 

1.2382 

1.1992 

1.2207 

1.1869 

1.1937 

Avg. correction 

1.2389 

1.1993 

1.2205 

1.1873 

1.1926 

Clm. correction 

1.2468 

1.1993 

1.2157 

1.1797 

1.1825 

EAA 

No correction 

1.2179 

1.1907 

1.2141 

1.1825 

1.1931 

Avg. correction 

1.2125 

1.1865 

1.21 

1.1787 

1.1888 

Clm. correction 

1.2354 

1.1945 

1.2146 

1.1814 

1.1904 

REG 

No correction 

1.3822 

1.2546 

1.2539 

1.2035 

1.1928 

Avg. correction 

1.3784 

1.2526 

1.2537 

1.2046 

1.1918 

Clm. correction 

1.4939 

1.291 

1.2645 

1.2073 

1.1915 

AVG 

No correction 

1.6645 

1.6783 

1.6983 

1.6896 

1.726 

Avg. correction 

1.3597 

1.3562 

1.3644 

1.3366 

1.3649 

Clm. correction 

1.2748 

1.2328 

1.2342 

1.1913 

1.219 

GEM 


1.2393 

1.1994 

1.2213 

1.1881 

1.1956 
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TABLE VIII: The zonal surface wind RMSEgaw for tho diffemntforecasters and bias correction methods. 
The climatology is included in the ensemble. 


Forecaster 

Bias correction 

Eearning period 



5 

10 

15 

20 

25 

EGA 

No correction 

1.6029 

1.5622 

1.565 

1.5288 

1.5642 

Avg. correction 

1.6004 

1.5643 

1.5636 

1.5365 

1.5722 

Clm. correction 

1.6666 

1.5962 

1.585 

1.5525 

1.5881 

EWA 

No correction 

1.6264 

1.5708 

1.5686 

1.531 

1.5684 

Avg. correction 

1.6251 

1.5712 

1.5685 

1.5324 

1.57 

Clm. correction 

1.6382 

1.5741 

1.57 

1.5347 

1.5718 

EAA 

No correction 

1.6016 

1.5622 

1.5634 

1.5305 

1.569 

Avg. correction 

1.6026 

1.5629 

1.5636 

1.53 

1.5679 

Clm. correction 

1.6356 

1.5732 

1.5693 

1.5332 

1.5708 

REG 

No correction 

1.682 

1.6037 

1.5887 

1.5486 

1.5819 

Avg. correction 

1.6844 

1.605 

1.5892 

1.5493 

1.5832 

Clm. correction 

1.7564 

1.6348 

1.6009 

1.5623 

1.5911 

AVG 

No correction 

1.8141 

1.8157 

1.8275 

1.8284 

1.8813 

Avg. correction 

1.6765 

1.6683 

1.6758 

1.6631 

1.7073 

Clm. correction 

1.7176 

1.6471 

1.641 

1.6092 

1.6541 

GEM 


1.6285 

1.5719 

1.569 

1.5323 

1.5692 
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TABLE IX: The meridional surfaee wind RMSEgaw for tho diff&mnt forecasters and bias eorreetion 
methods. The elimatology is ineluded in the ensemble. 


Foreeaster 

Bias eorreetion 

Eearning period 



5 

10 

15 

20 

25 

EGA 

No eorreetion 

1.3119 

1.2777 

1.282 

1.2587 

1.2707 

Avg. eorreetion 

1.3108 

1.2785 

1.2835 

1.263 

1.2748 

Clm. eorreetion 

1.364 

1.3025 

1.3002 

1.2769 

1.287 

EWA 

No eorreetion 

1.3339 

1.2848 

1.2874 

1.2607 

1.2737 

Avg. eorreetion 

1.3329 

1.2846 

1.2872 

1.2616 

1.2731 

Clm. eorreetion 

1.3411 

1.2874 

1.2885 

1.2627 

1.2742 

EAA 

No eorreetion 

1.3102 

1.2762 

1.2816 

1.2588 

1.2716 

Avg. eorreetion 

1.3124 

1.2779 

1.2829 

1.2594 

1.2717 

Clm. eorreetion 

1.3395 

1.286 

1.2875 

1.2621 

1.2738 

REG 

No eorreetion 

1.3744 

1.3068 

1.3023 

1.274 

1.2851 

Avg. eorreetion 

1.3753 

1.3075 

1.3028 

1.2754 

1.2861 

Clm. eorreetion 

1.4337 

1.3232 

1.3111 

1.2837 

1.2929 

AVG 

No eorreetion 

1.4671 

1.4692 

1.4776 

1.4779 

1.5038 

Avg. eorreetion 

1.3774 

1.3732 

1.3804 

1.3735 

1.3935 

Clm. eorreetion 

1.3996 

1.3427 

1.34 

1.3188 

1.3347 

GEM 


1.3352 

1.2853 

1.2876 

1.2617 

1.2731 
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TABLE X: The surface pressure RMSEgaw for tho differ&nt forecasters and bias correction methods. The 
climatology is included in the ensemble. 


Forecaster 

Bias correction 

Eearning period 



5 

10 

15 

20 

25 

EGA 

No correction 

2.5682 

2.465 

2.4716 

2.4189 

2.4999 

Avg. correction 

2.5228 

2.4518 

2.4667 

2.412 

2.5131 

Clm. correction 

2.6355 

2.4967 

2.4986 

2.4404 

2.5374 

EWA 

No correction 

2.5735 

2.453 

2.4607 

2.3942 

2.4938 

Avg. correction 

2.5712 

2.4528 

2.4603 

2.3942 

2.4942 

Clm. correction 

2.5833 

2.4569 

2.4631 

2.3978 

2.4976 

EAA 

No correction 

2.6987 

2.5417 

2.5194 

2.4457 

2.536 

Avg. correction 

2.5318 

2.4412 

2.454 

2.3912 

2.4932 

Clm. correction 

2.5821 

2.4552 

2.4614 

2.3951 

2.4957 

REG 

No correction 

2.7339 

2.5263 

2.5076 

2.4396 

2.5213 

Avg. correction 

2.7338 

2.5263 

2.5076 

2.4396 

2.5213 

Clm. correction 

2.8891 

2.5684 

2.5303 

2.4565 

2.5325 

AVG 

No correction 

5.9201 

5.917 

5.9363 

5.9402 

6.0273 

Avg. correction 

2.6251 

2.6026 

2.6277 

2.6045 

2.7222 

Clm. correction 

2.7146 

2.5816 

2.5876 

2.5354 

2.6441 

GEM 


2.5746 

2.4531 

2.4606 

2.3935 

2.4933 
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TABLE XI: The surface temperature STDqaw for tho difierent forecasters and bias correction methods. 


Forecaster 

Bias correction 

Eearning period 



5 

10 

15 

20 

25 

EGA 

No correction 

1.5722 

1.5516 

1.5509 

1.5003 

1.4699 

Avg. correction 

1.2071 

1.1681 

1.1419 

1.1 

1.0898 

Clm. correction 

1.1107 

1.047 

1.0066 

0.97441 

0.96212 

EWA 

No correction 

1.1803 

1.1255 

1.0967 

1.0734 

1.0492 

Avg. correction 

1.0505 

1.0143 

0.99385 

0.96995 

0.96079 

Clm. correction 

1.0001 

0.9584 

0.9334 

0.9094 

0.90286 

EAA 

No correction 

1.3797 

1.1943 

1.0777 

0.9866 

0.91267 

Avg. correction 

1.1392 

1.0059 

0.91539 

0.83884 

0.77893 

Clm. correction 

1.0596 

0.92525 

0.83163 

0.77086 

0.71868 

AVR 

No correction 

1.7773 

1.7817 

1.7849 

1.7888 

1.7888 

Year bias 

1.2573 

1.2335 

1.2129 

1.2 

1.1874 

Month bias 

1.1132 

1.053 

1.0151 

0.99245 

0.9729 
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TABLE XII: The zonal surface wind STD qaw for tho differentand bias correction methods. 


Forecaster 

Bias correction 

Eearning period 



5 

10 

15 

20 

25 

EGA 

No correction 

1.3211 

1.3083 

1.2938 

1.2958 

1.301 

Avg. correction 

1.218 

1.1991 

1.1895 

1.1955 

1.1906 

Clm. correction 

1.1857 

1.1267 

1.1117 

1.1121 

1.1013 

EWA 

No correction 

1.1648 

1.1222 

1.1094 

1.1066 

1.1125 

Avg. correction 

1.1275 

1.0924 

1.0866 

1.0938 

1.0903 

Clm. correction 

1.0902 

1.034 

1.0298 

1.0247 

1.0193 

EAA 

No correction 

1.2084 

1.0821 

0.98004 

0.91758 

0.86497 

Avg. correction 

1.1193 

1.0126 

0.92726 

0.88286 

0.83293 

Clm. correction 

1.0989 

0.96991 

0.88354 

0.83389 

0.7855 

AVR 

No correction 

1.3665 

1.3673 

1.3694 

1.3751 

1.377 

Avg. correction 

1.1962 

1.189 

1.1858 

1.1892 

1.187 

Clm. correction 

1.1566 

1.1034 

1.0871 

1.085 

1.0759 
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TABLE XIII: The meridional surfaee wind STD gaw for tho Aifiemnt forecasters and bias eorreetion 
methods. 


Foreeaster 

Bias eorreetion 

Eearning period 



5 

10 

15 

20 

25 

EGA 

No eorreetion 

1.0428 

1.0292 

1.0279 

1.0221 

1.0122 

Avg. eorreetion 

0.959 

0.95016 

0.94543 

0.94398 

0.93683 

Clm. eorreetion 

0.93743 

0.9019 

0.88682 

0.88115 

0.86831 

EWA 

No eorreetion 

0.91457 

0.88655 

0.87795 

0.87873 

0.86699 

Avg. eorreetion 

0.88297 

0.87172 

0.86361 

0.86215 

0.85405 

Clm. eorreetion 

0.85492 

0.83007 

0.81715 

0.80731 

0.79725 

EAA 

No eorreetion 

0.94936 

0.85194 

0.78035 

0.7294 

0.67994 

Avg. eorreetion 

0.88457 

0.80834 

0.74702 

0.70583 

0.66468 

Clm. eorreetion 

0.86813 

0.76843 

0.69957 

0.65128 

0.60696 

AVR 

No eorreetion 

1.0623 

1.0622 

1.0646 

1.0653 

1.0663 

Avg. eorreetion 

0.9454 

0.93813 

0.93673 

0.93579 

0.9352 

Clm. eorreetion 

0.92446 

0.88248 

0.86953 

0.86301 

0.85696 
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TABLE XIV: The surface pressure STD qaw for tho differentand bias correction methods. 


Forecaster 

Bias correction 

Eearning period 



5 

10 

15 

20 

25 

EGA 

No correction 

4.4105 

4.351 

4.3096 

4.2797 

4.2831 

Avg. correction 

2.4633 

2.4556 

2.451 

2.453 

2.4329 

Clm. correction 

2.4548 

2.3638 

2.3503 

2.343 

2.314 

EWA 

No correction 

2.5294 

2.3962 

2.3722 

2.3379 

2.327 

Avg. correction 

2.3429 

2.3491 

2.3509 

2.3472 

2.3384 

Clm. correction 

2.3384 

2.3005 

2.3073 

2.2899 

2.2663 

EAA 

No correction 

3.7348 

3.267 

2.9498 

2.7572 

2.5945 

Avg. correction 

2.2853 

2.1309 

2.0012 

1.9073 

1.8249 

Clm. correction 

2.2975 

2.0974 

1.9856 

1.8839 

1.7998 

AVR 

No correction 

5.8832 

5.885 

5.8841 

5.8911 

5.8787 

Avg. correction 

2.4017 

2.3929 

2.3839 

2.3834 

2.3636 

Clm. correction 

2.3771 

2.2727 

2.2459 

2.2339 

2.2015 
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TABLE XV: The surface temperature STD gaw for tho (Miemni forecasters and bias correction methods. 
The climatology is included in the ensemble. 


Forecaster 

Bias correction 

Eearning period 



5 

10 

15 

20 

25 

EGA 

No correction 

0.85636 

0.79338 

0.74092 

0.74888 

0.77041 

Avg. correction 

0.69405 

0.65224 

0.60268 

0.60546 

0.59947 

Clm. correction 

0.82139 

0.76932 

0.72812 

0.69949 

0.68463 

EWA 

No correction 

0.069617 

0.04385 

0.034291 

0.034327 

0.046811 

Avg. correction 

0.093904 

0.055604 

0.042137 

0.042974 

0.0541 

Clm. correction 

0.32004 

0.22664 

0.18245 

0.18477 

0.22204 

EAA 

No correction 

0.86891 

0.69606 

0.60843 

0.56257 

0.52853 

Avg. correction 

0.64118 

0.49176 

0.41823 

0.37824 

0.35085 

Clm. correction 

0.61502 

0.44528 

0.35476 

0.31043 

0.28258 

AVR 

No correction 

1.8474 

1.8482 

1.8513 

1.8555 

1.8556 

Avg. correction 

1.301 

1.2736 

1.2536 

1.2398 

1.2267 

Clm. correction 

1.1244 

1.0643 

1.0281 

1.006 

0.98709 
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TABLE XVI: The zonal surface wind STDqaw for tho differentand bias correction methods. 
The climatology is included in the ensemble. 


Forecaster 

Bias correction 

Eearning period 



5 

10 

15 

20 

25 

EGA 

No correction 

0.73276 

0.65502 

0.60472 

0.6103 

0.55895 

Avg. correction 

0.73511 

0.66343 

0.60299 

0.60858 

0.55592 

Clm. correction 

0.84851 

0.76309 

0.71594 

0.69938 

0.65361 

EWA 

No correction 

0.084994 

0.046667 

0.035807 

0.034519 

0.030238 

Avg. correction 

0.11876 

0.066557 

0.052488 

0.046515 

0.041827 

Clm. correction 

0.34414 

0.19312 

0.13497 

0.11729 

0.10056 

EAA 

No correction 

0.70173 

0.54873 

0.47246 

0.43713 

0.41112 

Avg. correction 

0.60517 

0.46204 

0.39078 

0.35748 

0.33391 

Clm. correction 

0.60982 

0.43764 

0.35328 

0.3177 

0.29133 

AVR 

No correction 

1.439 

1.4335 

1.4333 

1.4364 

1.4383 

Avg. correction 

1.2418 

1.2278 

1.2233 

1.2237 

1.2214 

Clm. correction 

1.173 

1.1189 

1.1023 

1.0993 

1.0906 
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TABLE XVII: The meridional surfaee wind STD qaw for tho differentand bias eorreetion 
methods. The elimatology is ineluded in the ensemble. 


Foreeaster 

Bias eorreetion 

Eearning period 



5 

10 

15 

20 

25 

EGA 

No eorreetion 

0.56695 

0.50983 

0.47015 

0.47185 

0.44052 

Avg. eorreetion 

0.56347 

0.52115 

0.47243 

0.47293 

0.43637 

Clm. eorreetion 

0.67315 

0.61536 

0.56948 

0.55546 

0.52289 

EWA 

No eorreetion 

0.056521 

0.031311 

0.025234 

0.025545 

0.022147 

Avg. eorreetion 

0.078621 

0.046682 

0.036763 

0.030673 

0.026266 

Clm. eorreetion 

0.26654 

0.15412 

0.11447 

0.094416 

0.080736 

EAA 

No eorreetion 

0.54687 

0.42964 

0.37097 

0.3416 

0.32204 

Avg. eorreetion 

0.48228 

0.36769 

0.31136 

0.28407 

0.26622 

Clm. eorreetion 

0.47982 

0.34436 

0.27938 

0.25008 

0.22985 

AVR 

No eorreetion 

1.1258 

1.119 

1.1194 

1.1178 

1.1187 

Avg. eorreetion 

0.98815 

0.9738 

0.97075 

0.96766 

0.96716 

Clm. eorreetion 

0.93731 

0.89451 

0.88127 

0.87434 

0.86855 
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TABLE XVIII: The surface pressure STDqaw for tho different forecasters and bias correction methods. 
The climatology is included in the ensemble. 


Forecaster 

Bias correction 

Eearning period 



5 

10 

15 

20 

25 

EGA 

No correction 

2.9392 

2.717 

2.5768 

2.5104 

2.3833 

Avg. correction 

1.5787 

1.5048 

1.4308 

1.3989 

1.2599 

Clm. correction 

1.7647 

1.631 

1.5915 

1.5789 

1.4751 

EWA 

No correction 

0.18324 

0.099206 

0.073743 

0.064644 

0.055238 

Avg. correction 

0.19148 

0.13322 

0.10974 

0.088419 

0.07355 

Clm. correction 

0.60822 

0.39215 

0.31246 

0.27342 

0.22431 

EAA 

No correction 

2.9241 

2.4551 

2.1702 

2.0292 

1.9133 

Avg. correction 

1.1266 

0.85932 

0.71971 

0.65848 

0.60953 

Clm. correction 

1.1816 

0.83857 

0.67345 

0.60363 

0.54823 

AVR 

No correction 

6.2919 

6.2863 

6.283 

6.2882 

6.2762 

Avg. correction 

2.4483 

2.4306 

2.42 

2.4176 

2.3972 

Clm. correction 

2.3963 

2.2911 

2.2639 

2.2515 

2.2187 
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